One aspect of the field of software testing focuses on the testing of graphical user interface programs (GUI programs). A GUI program is a program that provides functionality which is not accessible from a command line but is accessible through the program's graphical user interface. Typically, GUI program testing takes place during a product development cycle of a new program. GUI program testing may also take place during the development of a new version of an existing GUI program.
Due to the inadequacies of manual testing techniques, automated testing techniques have been developed. FIG. 1 illustrates a computer system 100 for testing a GUI program 101 using automated testing techniques found in the prior art. The computer system 100 includes a display device 103, a storage device 105, an input device 107, and a computer 109. The computer includes an interface 111, a processor 113, and a computer memory 115, which stores the GUI program 101. The GUI program includes a GUI component 119 and an engine component 121. The GUI component updates and retrieves information from a frame buffer associated with a GUI 117. In short, the GUI component has program code to control changes to the GUI. The code may be generated automatically from GUI builders. Engine code is code that implements the program's functionality. For example, engine code generates the information to be displayed on the GUI.
Prior art automated testing methods simulate input events (such as mouse clicks) in order to test the same data processing paths that are invoked in response to user input. In this way all functionality in a GUI program is tested through the program's GUI component. The prior art methods begin the testing process by having a user enter data through a GUI 117 using the input device 107. For example, the inputs may indicate that a "ButtonPress" event should be processed. The method records the user inputs and saves them to a separate file. The recorded inputs are later replayed as a test script 123. A test script engine 124 drives the replay of the test script 123. In response to the inputs, the GUI 117 changes its state. For example, if the inputs indicate that a "ButtonPress" operation should be invoked, then the method performs the steps associated with the ButtonPress event. In response to the changes in the GUI the method saves the new GUI state as a bitmap in a raster file (also called saving a "snapshot" of the GUI). The bitmap may also be saved in any other type of image format. In addition to or in place of taking a snapshot the method may perform resource verification for the widgets displayed on the GUI.
The snapshot acts as a baseline for comparing the program's response to user inputs at an earlier point in the development process with the program's response to the same user input at a later point in the development process. For example, as the development process continues, changes are made to the program code underlying the GUI component 119 and the engine component 121. In order to test the changes using automated techniques, the prior art method replays the input events recorded earlier. In response to replaying the input events, the GUI component 119 invokes the engine component 121 which in turn instructs the GUI component to redraw the GUI. In this way the engine component 121 is invoked through the GUI component 119. The method then takes a snapshot of the changes to the GUI 117. Finally, the earlier snapshot is compared to the later snapshot. If the snapshots are identical then the method concludes that the changes made to the program work properly. Non-identical snapshots may indicate that a bug has been introduced into the program.
One of the problems associated with automated testing techniques involves properly synchronizing the replay of the input events. Problems arise because there is often an interdependence between the input events. For example, assume a first input event displays a menu on the GUI and a second input event selects an entry on the menu. For the second event to work properly, the menu must be displayed before the second input event is executed. If the computer tries to select an entry on a menu that is not displayed, the system will return an error.
In the past some system developers have tried to address the synchronization problem by inserting delays into their test scripts. Continuing with the example above, these developers would insert a delay command between the first and second input events in the hopes that the delay would be long enough to ensure that the "menu selection event" would be executed after the system had completed displaying the menu.
Inserting delay commands into test scripts often fails to solve the synchronization problem because the total amount of time it takes a given event to execute on a particular computer varies each time the computer executes the event. For at least this reason time delays are inadequate to ensure proper synchronization between events.
The variation in the amount of time it takes a given event to finish executing becomes even more pronounced when the comparison is made between events executing on different systems. Such a case arises when a test script is used to test a port of a particular software product. For example, if a software developer ports an operating system from a platform with a faster microprocessor to a platform with a slower microprocessor then the time delay originally inserted into the test script may not be sufficient to ensure proper synchronization. For at least this additional reason, time delays are inadequate to ensure proper synchronization.
Furthermore, benchmarking the performance of a software product is impossible if time delays are introduced into the test script. Benchmarking involves running a software product through a series of tests and measuring the performance of the software product during the tests. Obviously, if time delays are introduced into the test script, the measurements from a benchmark become meaningless. For at least this additional reason time delays are undesirable.
Introducing time delays poses a general problem, which is that they slow the test execution process down. This problem becomes more pronounced when using large test suites.
It would be beneficial to develop an improved method and system for ensuring proper synchronization between test events.